fix(kernel): preserve empty metadata filters - #933
Conversation
![peco-review-bot[bot]](https://avatars.githubusercontent.com/in/3801177?s=60&v=4){kind=small}
There was a problem hiding this comment.
Verdict: 1 Medium
Clean, well-scoped change that stops collapsing empty/blank metadata filters to None and preserves them as real (match-nothing) patterns, with docstrings/CHANGELOG/tests updated to match. One medium concern: the correctness of the new "empty string matches nothing" behavior rests on kernel semantics that the removed comment described as the opposite (kernel rejecting "" with InvalidArgument), and the only tests exercising it are live-warehouse e2e tests — worth confirming against the real kernel and checking whether the ^0.2.0 pin needs bumping.
Signed-off-by: Vu Anh Phung <vu.phung@databricks.com>
Signed-off-by: Vu Anh Phung <vu.phung@databricks.com>
79b87d7 to
74d883f
Compare
There was a problem hiding this comment.
Verdict: 1 Medium
Focused, well-tested fix that flips empty metadata filters from match-all to match-nothing. One medium concern: get_tables passes an empty catalog straight to the kernel while get_schemas/get_columns adapt it via _exact_catalog_and_pattern, and no test verifies tables(catalog_name="") actually matches nothing — worth confirming the kernel's list_tables doesn't treat blank as "all catalogs."
GitHub Actions
Failure
There was a problem hiding this comment.
Verdict: 1 High · 1 Low
One real concern: get_tables is the odd one out — it passes catalog=catalog_name unbridged while get_schemas/get_columns route empty catalogs through the new _exact_catalog_and_pattern helper, so tables(catalog_name="") likely matches all catalogs (or raises) instead of "matches nothing" as the PR's own docstring promises (F1, high). Test coverage for the empty-catalog tables case is also missing (F2, low). The schema/table/column pattern preservation and the empty-catalog bridge for schemas/columns look correct.
There was a problem hiding this comment.
Verdict: 1 Low
Looks good — a focused, well-tested behavior change that stops collapsing empty/blank metadata filters to None and forwards them to the kernel unchanged. Unit + e2e coverage is thorough and no references to the removed _none_if_blank/_catalog_or_none helpers remain. One low-severity docstring-consistency nit around how catalog_name is described across the three metadata methods.
There was a problem hiding this comment.
Verdict: 1 Medium · 1 Low
Focused, well-tested change that stops collapsing empty/whitespace metadata filters to None on the kernel path. Two concerns: (1) the "empty pattern matches nothing" contract now depends on kernel behavior the connector no longer controls — the removed helper documented that the kernel rejects "" with ProgrammingError, and mocked unit tests can't detect a mismatch, so please confirm/pin the kernel version; (2) the shared abstract base-class docstring now documents kernel-only %/* catalog semantics that don't hold for the Thrift backend.
| stream = self._kernel_session.metadata().list_schemas( | ||
| catalog=_catalog_or_none(catalog_name), | ||
| schema_pattern=_none_if_blank(schema_name), | ||
| schema_pattern=schema_name, |
There was a problem hiding this comment.
🟡 Medium — The change now forwards empty-string pattern filters (schema_pattern, table_pattern, column_pattern) to the kernel verbatim instead of collapsing them to None. The PR description, the base/Cursor docstrings, and the e2e tests all assert that an empty pattern matches nothing (fetchall() == []).
But the previous helper's own docstring documented the opposite kernel behavior: Identifier/LikePattern reject "" with InvalidArgument, which the connector maps to ProgrammingError. If the pinned kernel (databricks-sql-kernel = "^0.2.0", pyproject.toml:62) still rejects "", then empty-string filters now raise ProgrammingError at runtime rather than matching nothing — a regression relative to the documented/asserted contract.
The unit tests (test_get_schemas_preserves_empty_pattern, etc.) mock the kernel session, so they only verify the connector passes "" through — they cannot detect that the real kernel rejects it. Only the real-wheel e2e tests would, and those run in a separate CI step. Please confirm this depends on a coordinated kernel change that accepts "" as match-nothing, and bump the minimum databricks-sql-kernel version (^0.2.0) accordingly so a user on an older kernel doesn't silently get ProgrammingError where the docs promise an empty result set. Applies equally to the call sites at lines 945 and 975-977.
(Anchored to the nearest changed line — see the description for the exact location.)
| catalog_name: Optional catalog name pattern to filter by | ||
| schema_name: Optional schema name pattern to filter by | ||
| catalog_name: Optional exact catalog name. ``None`` leaves the | ||
| filter unset; ``%`` and ``*`` select all catalogs; an empty |
There was a problem hiding this comment.
🔵 Low — This docstring lives on the abstract DatabricksClient base class, which is the shared contract for both the Thrift and kernel backends. It now states catalog_name is an "Optional exact catalog name" where "% and * select all catalogs." That semantics is kernel-only: the Thrift backend passes catalogName=catalog_name straight through (thrift_backend.py:1160/1206/1254) with no %/* normalization, so on Thrift % is a literal catalog name — as the removed _catalog_or_none comment itself noted ("This intentionally diverges from raw-Thrift literalness (Thrift treats % as a literal catalog name)").
A reader of the base contract (and Thrift users) will be misled into thinking catalog_name='%' matches all catalogs on every backend. Consider scoping the wildcard note to the kernel backend, or clarifying that it is a kernel-specific normalization. The same wording appears at databricks_client.py:332 (get_columns) and in the public Cursor docstrings at client.py:1583-1584 and 1641-1642, which are likewise backend-agnostic and user-facing.
1 participant
Fixes PECOBLR-4221.
The kernel metadata adapter no longer collapses empty or whitespace-only filters to
None. Empty pattern filters therefore match nothing, while exact filters retain the kernel's validation behavior. Existing%/*catalog wildcard normalization is unchanged.Testing: 265 focused unit tests passed;
git diff --checkpassed.